Transceiver Design for MIMO Systems with Improper Modulations

This paper considers joint transceiver designs for single-user multiple-input, multiple-output systems employing improper constellations such as binary phase shift-keying and M-ary amplitude shift-keying (M-ASK). Proposed are novel joint linear transceivers that minimize the total mean squared error of the symbol estimation at the output of the decoder. The joint linear transceiver designs are carried out for both cases of perfect channel state information (CSI) and imperfect CSI at the transmitter and receiver. For the case of imperfect CSI, the channel model takes into account both transmit and receive correlations as well as the channel estimation error. The superiority of the proposed transceivers over the previously-proposed designs is verified by simulation results.